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ABBREVIATIONS 


A/C 

ACARS 

ADI 

AFDS 

ALT 

APP 

APU 

A/T 

ATC 

C/B 

CDU 

CMD 

CRS 

CRT 

CWS 

EADI 

EEC 

EF1S 

EHSI 

EICAS 

ELEV 

EPR 

FAR 

FD 

FLCH 

FMC 

FMS 

FMST 

FO 

GPWS 

HDG 

HLD 

HSI 

HUD 

IAS 

ILS 

INS 

LNAV 

LOC 

MCP 

RWY 

SID 

SPD 

THRTL 

TMC 

TO 

VMC 

VNAV 

VOR 

V/S 


Aircraft ' 

Automated Communications Addressing and Reporting System 
A.ttitude Director Indicator 
Autopilot Flight Director System 
Altitude 

Approach Mode of AFDS 
Auxiliary Power Unit 
Autothrottle 
Air Traffic Control 
Circuit Breaker 

Control /Display Unit (Flight Management System) 

Command Mode of AFDS 
Course Direction 
Cathode Ray Tube 

Control Wheel Steering mode of AFDS 
Electronic Attitude Director Indicator 
Electronic Engine Control 
Electronic Flight Instrument System 
Electronic Horizontal Situation Indicator 
Engine Indication and Crew Alerting System 
Elevator 

Engine Pressure Ratio 
Federal Aviation Regulations 
Flight Director 

Flight Level Change mode of AFDS 
Flight Management Computer 
Flight Management System 
Flight Management System Trainer 
First Officer 

Ground Proximity Warning System 
Heading (also a mode of AFDS) 

Hold 

Horizontal Situation Indicator 
Head Up Display 
Indicated Air Speed 
Instrument Landing System 
Inertial Navigation System 
Lateral Navigation (mode of FMS) 

Localizer 

Mode Control Panel 
Runway 

Standard Instrument Departure 
Speed (a mode of the AFDS) 

Throttle 

Thrust Management Computer 
Takeoff 

Visual Meteorological Conditions 
Vertical Navigation (a mode of the FMS) 

VHF Omnidirectional Radio Range 
Vertical Speed (also a mode of the AFDS) 
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SUMMARY 


New cockpit technology is continually required for the airlines to remain competitive, and the 
manufacturers respond to this need. A historical view of the introduction of new technology 
suggests that the changes have not always gone as planned, and that there have been reactions 
to the new technology that were not anticipated. This report describes the first phase of a 
joint airline/NASA study which was undertaken during the introduction of a new technol- 
ogy aircraft, the B-767. This first phase had several objectives: to identify any adverse reac- 
tions to the new technology should any develop (none was found); to provide a "clearing 
house" of information for the airlines and pilots on experiences during the introductory 
period; to provide feedback on airline training programs for the new aircraft; and to provide 
field data to NASA and other researchers to help them develop principles of human interaction 
with automated systems. 

Three airlines and their pilots agreed to participate in the study. Data were obtained through 
more than 100 questionnaires returned by pilots, the direct observation and interviews with 
pilots and check airmen, and attendance by a NASA observer at the ground schools of the par- 
ticipating airlines. 

There are two points concerning the results that deserve particular emphasis. First, the data 
were taken during the early introduction of the aircraft and the conclusions apply only to 
that period. Second, although the B-767 was the only aircraft in the study, discussions with 
operators of the A310 (another uew-technology-cockpit aircraft) have confirmed very similar 
experiences. Thus, the following conclusions, while specifically mentioning the B-767, arc 
likely to be valid for the introductory period of the A310: 

1. Most the pilots enjoy flying the B-767 more than they enjoy flying the older air- 
planes. 

2. The pilots accept the new cockpit technology, and they choose to use it because 
they find it useful. 

3. The pilots are aware of the possible loss of flying skill with the presence of auto- 
mation, and they hand-fly (usually with flight director) to prevent this loss. The 
data collected in this study do not indicate any loss of skills. 

4. The primary points of confusion or surprise were autothrottle/autopilot interac- 
tions; the autopilot turning the "wrong way" or not capturing the course; and 
achieving desired results with the Flight Management System/Control Display 
Unit (FMS/CDU). 

5. The pilots felt training for the FMS/CDU could be improved, and they especially 
wanted more "hands on" experience. More training on the mode control panel, and 
more hand flying were also mentioned. 

6. Information, especially "techniques," may not always be getting from the system 
designers to u.e line pilots. 

7. Flying any aircraft with sophisticated equipment and high levels of automation 
allows distractions that cause a loss of monitoring performance. 


PRECEDING PACE BLANK NOT FILMED 
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8. Many pilots should be trained to "turn it off" and not try to "program" their way 
out of an anomalous situation. 

9. These field data confirm some existing human factors principles, suggest a new 
principle, and raise questions requiring further research. 
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INTRODUCTION 


Background 

V' 

New aircraft technology is continually required for the airlines and manufacturers to remain 
competitive. Most of the time the new technology takes the form of small, "add-on" systems 
to existing aircraft such as the Automated Communications Addressing and Reporting System 
(ACARS) or Ground Proximity Warning System (GPWS). infrequently, there is a dramatic 
change in cockpit technology, as with the introduction of the B-767 and Airbus A310. 

The Operators' View 

Based on previous experience with new technology, it was expected that there would be con- 
comitant changes required in the role of the crew, piloting techniques, procedures, and train- 
ing. It was generally pc.ceived that previous conversions to new technology did not always 
go smoothly; that many airlines experienced higher-than-expected training costs; and that 
some pi.ots had expeiienced difficulty in the transition to the newer wide-body jets (the L- 
1011 and DC-10). There have been several explanations offered for this; certainly, the flight 
guidance systems on these aircraft are more complex than those of their predecessors, but it 
has also been noted that the captains transitioning to these aircraft had not been to school in 
periods of 10 to 15 yr, and this may have contributed to some of the difficulties. 

As new technology in any field is developed, there are some events that were seemingly not 
anticipated by the designers. The GPWS, although admittedly introduced into service before 
many felt it was ready, has caused pilots to turn it off because of the high false-alarm rate of 
the system (Wiener and Curry, 1980). Subsequent changes in the alarm logic and display 
logic have modified this this situation substantially. The Inertial Navigation System (IN’S) pro- 
vides another example. It is true that, as automatic navigators, they navigate more accurately 
and more economically than manual navigators, but the class of navigation errors has changed 
so that a measurable fraction of errors occur due to the insertion of incorrect data and/or 
movement of the aircraft while the INS is aligning itself. Both types of incidents have caused 
aircraft to takeoff only to have to return to the airport because of these . nappropriate actions. 

A third example of an unanticipated side effect of automation has beet observed by the air- 
lines when pilots transitioned from first-officer on a wide body aircraft, with significant levels 
of automation, to upgrade to captain on a narrow-body aircraft, with less automation (Wiener 
and Curry, 1980). At first, there was a higher than expected failure rate, but this has dimin- 
ished after pilots started preparing themselves before the transition by performing more 
manual flying on the wide body aircraft. Many pilots have heard of others’ experience in this 
area and have altered their own use of the autopilot to avoid the apparent loss of skills. 

The Human Factors View 

In many respects the technology of human factors has not kept pace with the technology of 
the cockpit. There is a significant body of knowledge on how to design displays and controls 
— material on which manual systems are based — but there is little material to help the 
human-factors practitioner with the design of interfaces to complex devices. It has been felt 
by many observers that the performance of such systems will be determined less by tradi- 
tional manual piloting skills, but more by the pilots’ decision making behavior (what mode 
should I use?); their knowledge of the systems (is this thing working correctly?); their moni- 
toring behavior (keystrokes entered now may influence the system 5 hr later); and crew coor- 
dination (setup and monitoring of the systems and other members of the crew). 
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The job of the systems designer and operator is made even more complicated since many out- 
comes of the design and operation (such as the loss of skills described earlier) do not emerge 
until a considerable amount of experience with the new equipment has been gained. This is 
precisely the type of information that cannot be obtained in simulation, the traditional design 
tool. 

0 

In short, new human-factors techniques are required to assist in the design of new cockpit 
technology. 


Study Objectives 

The objectives of the joint airline/NASA study were as follows: 

1. To identify any unanticipated side effects of the new technology. 

2. To provide feedback to the carriers on their training related to the new cockpit 
technology. 

3. To help the exchange of operational experience among carriers. 

4. To provide quantitative data on the human-factors aspects of the new technology. 

5. To provide field study information for later development of human-factors "princi- 
ples" of automation. 
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DESCRIPTION OF THE STUDY 


The study was conducted with the help and cooperation of hundreds of other individuals 
within the three participating airlines. The major sources of information used in the study 
are outlined in this section. 


Ground School 

The NASA observer attended the f uli (2 wk) ground school of one airline, and 1 week periods 
at each ground school of the other two airlines; these periods coincided with instruction of 
Flight Guidance, instrumentation, and the Flight Management System. The observer did not 
take the oral exam or any simulator training, but he did observe three 4-hr simulator training 
sessions. 


Pilot Volunteers 

Pilot volunteers from the three participating airlines were solicited from those who attended 
767 transition training. A procedure was established with the carriers whereby the anonym- 
ity of each pilot would be preserved by having him adopt an identity code number. This was 
necessary to establish identification for a possible second round of questionnaires. Invitations to 
participate in the study, including a five-page question-and-answer booklet, were prepared for 
each airline. Initially the invitations to participate were distributed when the pilots enrolled 
in the ground school for transition training. Later this wrs changed so that the pilots received 
material after their simulator training, either before or jut-C after their initial operating experi- 
ence. 


Questionnaire 

The primary data-collection device was the questionnaire (see appendix A). Over 100 returns 
were received and 102 were used for most of the analyses. The questionnaire consisted of 
three parts: 


Frequency-c f -Use T able 

This part was designed to to determine what features were being used by the pilots, and how 
frequently they used these features. 

Open-Ended Questions 

The open-ended questions were designed to obtain information on confusing aspects of the sys- 
tems; the features and systems that the pilots like and find useful; characteristics that they 
don’t like; the aspects of the cockpit they would change if they could; and their opinion about 
the training they received. 
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Attitude Survey 


This portion or the questionnaire consisted of 36 statements about the pilots' opinions on auto 
mation and flying in general, and the airplane in particular; the pilots responded on a five- 
point "agree— disagree" Likert Scale. 

e 


Interviews and Meetings 

Informal interviews were held with approximately 20 pilots and 8 check pilots. Each inter- 
view lasted from0.5 to 1.5 hr. hours. 

Progress report meetings were held at each of the three participating airlines. Attendees of 
these meetings consisted of representatives from flight operations management, training, line 
pilots, and check airmen. These progress reports seemed to have a catalytic effect, since they 
always evolved into a spirited discussions among all attendees. 


Cockpit Observation 

The NASA observer flew as cockpit observer on one training flight (two pilots received train- 
ing on this flight), two segments during which a captain was receiving line training, and 
approximately 40 segments with line pilots operating the aircraft in normal line operation. 


Internal Documentation 

The airlines made available any pilot reports of irregularities or incidents that occurred. 
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RESULTS 


Questionnaires 


Respondents 

A total of 104 questionnaires had been received between February 22, 1983 and July 31, 1983 
(the cutoff date for the analysis). Two of the questionnaires could not be identified with a 
specific airline, so they have not been included in the analysis. The distribution of responses 
by airline, position (captain/first officer), total flying time, and time in the 767 is shown in 
Table 1. An interesting fact is that a majority of the respondents were captains, whereas our 
past experience has been that first officers are usually more likely to participate in studies of 
this type. 

Frequency-o f -Use T able 

The frequency-of-use table was distributed in two forms (see Appendix A) because of some 
ambiguities in the instructions and some apparent inconsistencies in the responses. These 
inconsistencies make it difficult to draw conclusions from these data alone, but these data are 
useful for confirming results suggested by other sources. See Appendix B for details. 

Open-ended Questions 

Without a doubt, the answers to the open-ended questions were the most difficult to extract 
and summarize, but they yielded extremely useful information. Included in this category of 
responses were any notations from the comment column of the frequency-of-use table, or 
comments from the pilot opinion portion of the questionnaire. These additional comments 
were solicited, and were quite useful. 

After approximately 30 or so questionnaires were carefully examined, several categories of 
response began to emerge. The responses to the open-ended questions are shown in Table 2, 
and have been grouped into Features Liked. Features Missing or Not Liked, Points of Confusion 
or Surprise, and Training. Not included in these responses are those comments relating to 
human engineering and cockpit environment issues, or comments regarding the implementa- 
tion of a particular feature if they were not pertinent to the present study. 

The narrative responses below are complete. They are brief because an essay response was not 
requested, nor was space provided. Nonetheless, they do convey the pertinent information. 

Features Liked The pilots felt positively toward the Autopilot Flight Director System 
(AFDS) and Autothrottle (A/T) System. When asked what feature they liked about the AFDS 
and autothrottle system, the pilots’ responses involved the general concept of autothrottle and 
speed control 

"A/T, saves fooling around setting power" 

"speed control without constant monitoring" 

"reliability and flexibility (ie., variety of ways of achieving an objective" 
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"automatic changeover to .80 rnach from 3CK) knots and vice versa" 
"throttles very handy in terminal area" 

"all" 


They also liked the AFDS and Thrust Management System during takeoff when the possibil- 
ity of overboosling the engines is negated by the Electric Engine Control (EEC). Also men- 
tioned were the reduced workload, altitude-capture, and altitude-select features: 


"like TO power feature" 

"TO and climb" 

"with EEC the ability to keep engine at proper N1 without having to set power 
manually" 

"autothrottle overpower protection" 

"altitude capture at preset speed on descent" 

"reduced workload; correct power is set" 

"saves time and effort setting different thrusts" 

"enables the pilot to narrow attention pattern when necessary to concentrate on 
most important objectives" 

"ease of operation with reduction in workload 


The Electronic Flight Instrument System (EFIS), or Attitude Director Indicator (ADI) and 
Horizontal Situation Indicator (HS1) Cathode Ray Tube (CRT) displays received enthusiastic 
response. Most of the comments were general in nature, and referred to the information on 
the displays and the clarity of the displays 

"I can't describe it, but there is something visually pleasing about the CRT 
presented instruments and Flight Director" 

"Easy to read, and all info readily available" 

"Very bright— always know where you are" 

"Good display— easy to read, and a wealth of information' 

Specific mention was often made of the map display 

a 

"The map mode and the HSJ is a wonderful tool" 
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"Page missing from available version" 



'multitude of capabilities' 
y works well, useful" 

"total amount of information is great" 

"ability to%urn my map!" 

In addition, two pilots specifically pointed out the advantage of having the route displayed on 
the CRT map dislay, a combination of the EFIS and FMS capabilities. 

"being able to string out the route with few entries" 

"ability to build a map presentation" 

E1CAS The Engine Indicating and Crew Alerting System also received many favorable, if 
general, comments about the quality and quantity of information on the display. 

"good, easily scanned displays, readily seen from any seat/cockpit position; excellent 
alerting system" 

"warning and annunciation of practically everything" 

"immediate info on status of airplane" 

"clarity of and ease of reading the displays" 

"much attention in small space; warning system gets attention" 

"quick glance comprehension of A/C system status" 

Specifically mentioned by several pilots were the explicit display of engine limits as well as 
the ability to monitor a large number of variables. 

"anti-ice & exceedence displays and TMC [thrust management computer] combina- 
tion" 

"limit displays" 

"all engine limits well displayed; no numbers to remember" 

"monitoring capability" 

"cautions and warnings" 

"annunciations for prompt attention" 
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* ...engine displays and controls" 

"constant monitoring of systems" 

"alerts crew to any malfunction, and [l] like the call system (red, yellow)" 

"the needle style of display catches the eye when something changes [compared to] 
digits" 

"color changes with warning/caution" 


Features Not Liked or Missing There were not many features relating to the present study 
that the pilots either did not like or felt were missing. An almost traditional complaint of 
computer users is the slow response time, and these pilots were no exception. Usually, the 
complaint concerned their time in the terminal area where they perceived fast flight crew 
response as a necessity due to ATC changes: 


"sometimes slow to accept information and update display" 

"they didn’t make it sophisticated enough— it is too slow" 

"sometimes takes too long to bring up system and execute it" 

"difficulty and length of time to reprogram approach once it is activated" 

"seems a bit cumbersome at times— making changes for approach (as controllers are 
prone to do)" 

"dislike being unable to change approaches quickly; need a way to clear out old 
approach in one step, so that new one can be inserted quickly" 


Pilots from airlines that did not use mechanical checklists felt they would be useful, and 
many thought a checklist should be on the EICAS. 


"checklists, emergency and irregular procedures should appear in conjunction with 
key events (e.g., gear down, engine failure, etc.)" 

"checklist on control column plate holder" 


Others felt that t:ie the circuit breakers and spare light bulbs should be within reach of the 
pilots: even though the design philosophy precludes the necessity of needing these, the exigen- 
cies of line operation made the pilots think otherwise 


" have spare bulb box within reach when seated" 


- 13 - 


"place circuit breakers in place where one pilot didn't have to get out of his seat to 
reset" 

"the most important (cockpit change} is to bring ell controls spare bulbs, C Ils. etc. 
to within reach of pilots ... very important for two man crew" 

"move the spare bulbs where 1 can get to them. Way in back is no good on final 
with a blank gear down indicator’ 

"Wc had buth lights in the right main gear out on final approach and had to go 
around while the FO got out of his seat and found bulbs in the dark. Move spare 
bulb supply or have a second supply." 


Points of Confusion or Surprise The first three General Questions yielded most of the 
responses relating to points of confusion or surprise, especially question number 2. Only the 
more significant categories w'ill be described here. 

Autothrottle-V/S-SPD I nler action A significant number of pilots repo/ tea confusion on 
the interaction of pitch autopilot and autothrottles. 

"seems easy to turn off autothrottle intentionally and then get it back by getting 
into a speed mode and not realizing it at once" 

"sometimes autothrottles reconnect when not expected to, even though they are 
working normally" 

"occasional misunderstanding of FLCH capability with [autothrottle] turned off" 
"cannot always obtain zero thrust; why?" 

"some confusion as to when A/T will reengage after manual disconnect" 

"trying to lose altitude with speed brakes, and then throttles power up the 
engines" 

"once aircraft leveled at uncalled for altitude, and autothrottle did not respond’ 

"(have observed] confusion between SPD on A/T and SPD on pitch, and many simi- 
lar problems" 

"descending in CMD, V/S, throttles at idle, autothrottles disconnected but armed. 
[Mode control panel] airspeed is well below existing speed. When 1 select SPD the 
autothrottles come out of idle. As far as 1 am concerned, this is not logical. Result; 1 
disconnect A/T." 

"interface between V/S and SPD is bad— won’t let throttles come back to idle and 
then [throuies are too] slow to react." 

"Autothrottle is difficult to use properly, particularly in use with descents. When 
manual throttles are used, they stay wherever you put them. With A/T, it is 
necessary to constantly check power which actually increases workload." 
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Speed Sync at FLCH Engagement A seemingly related comment is the speed synchroni- 
zation at the time I'llpht Level Change (FLCH) is engaged. Regardless of the value displayed 
in the speed-select window at the time FLCH is engaged, this displayed value is changed to 
the existing speed and the pitch autopilot holds the existing IAS. The autothrottles advance to 
maximum allowable thrust (if climbing) or reduced thrust (if descending). Most pilots who 
reported this confusion felt the target speed should have been that which was displayed in 
the window at the time of engagement. 

Aircraft Turns " Wrong Way " or Does Not\Capture Pilots reported that while on autopi- 
lot the airciaft turns the wrong way, especially upon localizer intercept or after crossing a 
waypoint or that the LNAV system did not engage at all. 

“twice approaching XXX Rwy ILS12; once the aircraft leveled at uncalled for alti- 
tude, and autothrottle did not respond; second time, aircraft began non-specified 
climbing right turn after tracking in " APP" mode on ILS (autothrottle was inop).“ 

“ hard pitch down in terminal area could not be explained." 

"autothrottle (l) took off at flare; (2) started [go-around] at 400’ on approach" 

"approach to YYY ILS22R: aircraft turned about 40 degrees right of approach 
course just prior to [outer markerj_no apparent reason for malfunction." 

"_the autothrottles occasionally don't engage during TO and rlimb; the automatics 
occasionally don’t properly lock on during localizer capture." 

"when using the direct/intercept, a number of pilots, including myself, have for- 
gotten that the active waypoint must be ahead, not behind. Aircraft will not 
intercept desired course. In similar situation, aircraft will turn lack to active 
waypoint behind, (if you let it). This has ’surprised’ several of us." 

"We were cleared direct ZZZ. We used the fix key to define all of the abeam way 
points. We activated the route. Subsequently, we were given an off course vector 
for traffic. When we were again cleared on course, we did not pass within 2.5 
miles of WWW', so the aircraft started to go back to WWW." 

"Enroute New York over TTT, [the FMS] suddenly drew a perfect 360 degree cir- 
cle and immediately started to turn." 

"twice when in approach mode at YYY, the autopilot tunt to intercept localizer 
when 5 miles or more to go. When reset on approach mcx actually approaching 
localizer, flew through completely." 

"Have seen A/P ’capture’ an ILS while still at least 5 miles away. Have also seen 
A/C start to turn the wrong way to capture a radial." 

"Twice the heading select, when activated, has begun a turn in the wrong direc- 
tion." 

"Locked onto LOC, followed by disengage and turn away from LOC course." 
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"Had the aircraft start a turn at a waypoint when it should not have while in 
LNAV. Probably was an old route still in CDU that was not properly erased." 

"In LNAV, airplane started out on excursion for no known reason— it happened 
twice, both at waypoints." 

"Unexplained turns away from the LNAV course magenta line. Also no capture of 
certain departure SIDs although they were adequately displayed." 

An obvious drawback of voluntary' reports such as this is that it is not always possible to iso- 
late the cause of these events, e.g, system setup error, system malfunction, or incorrect 
knowledge of the system itself. The same situation exists with the reports about unselected 
mode changes. The following reports are typical: 


"had MCP switch from FLCH to V/S 3 times." 

"the system has switched from FLCH to vert end. twice." 

It is likely that the events happened as reported, but it it is also difficult to isolate causes from 
such brief reports. 

Heading Display on Track Up Map Eleven pilots reported confusion between the head- 
ing orientation and the track up nature of the map display; as they describe it, this almost 
always occurs during vectoring in the terminal area when they are controlling aircraft head- 
ing, not track. 


"Flying a HDG, when initially checking out on A/C when in map mode." 

"Heading info is hard to get used to. All of us have trouble finding heading info." 

"Track up’ presentation disconcerting when trying to maintain heading manually." 

"Pilots fiv using heading, engineers use track. Heading should be prominently 
displayed at top of map." 

T raining 

Four of the pilots felt their training was adequate and did not require any changes. 

FMS/CDU The great majority, however, had at least a few comments to make, espe- 
cially regarding the FMS/CDU and the type of training desired. 

"[more] use of CDU, i.e, ’rules’ of CDU— what it will and won’t’ do, using all the 
different ways to get a job done" 

"[more] CDU. CDU. CDU_the simulator wasted incredible time because no decent 
CDU training device [was available]." • 

"more emphasis on FMC; need FMC trainer* 
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"without a doubt, the FMS and CDU should receive much more emphasis." 

"More practice and hands on a fully operational 'identical' FMS/CDU and practice 
to full proficiency." 

"...Insight into the capabilities of the [FMS/CDU] systems would be better obtained 
through an actual iata-based trainer that operated in real time and allowed trial 
and error: Inputs/Mistakes/Corrections/Leaming!" 


Related comments referred to the specific exercises that they felt would be useful: 

"[more] CDU use in simulator with emphasis on everyday line use combined with 
typical line ATC clearances." 

"A little less I^iC— or a more practical approach." 

"Less emphasis on VN'AV (since it’s not installed), more emphasis on CDU program- 
ming." 


AFDS Training Several of the pilots would have liked more training -with the AFDS 
and Mode Control Panel. 


“The simulator should establish basic flying skills with the AFDS prior to using 
the automatics." 

"[more] operation of all automatics. A hands-on mockup is needed in ground 
school." 

"[more] autoflite system. If procedures for use were drilled slowly in a step by 
step fashion, particularly during TO profile and approaches, if these procedures 
were down cold before simulator training, simulator would be much easier." 

’[An] absolute necessity [is] basic AFDS training! Why do we have this autopilot? 
What is it trying to do? What is the design philosophy? Needed after this: DRILL! 
DRILL! DRILL! with no other simulator movement, just AFDS...." 

"More line oriented crew duties (log book, set up, comm, etc.)" 


Additional Flying A significant number of pilots wished to have more experience hand 
flying the simulator, and several suggested the order of presentation of the material. 


"Hand flying needs more emphasis. Total time should lie increased. Co-pilots should 
get equal time." 

"More manual flying. This airplane will fly just fine without the AFDS and 
autothrottles." 
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Pi'ol Opinion Questionnaire 


The pilots responded to 36 statements and were asked to circle one of five answers to describe 
how they felt about the statement: strongly agree: slightly agree: neither agree nor disagree; 
slightly disagree; or strongly disagree. Their responses were examined to determine if there 
was any correlation with the following variables: airline, total flying time, flying time in the 
767, and their position (e.g., captain or first officer). In addition, a factor analysis was per- 
formed to determine if there were any underlying dimensions to the response to the 36 ques- 
tions. 

All responses were pooled and the results appear in Table 3 for each of the 36 questions. 

Airline Differences A contingency table analysis was first performed to determine whether 
or not any gross differences existed between airlines. The responses were pooled into a 3 X 3 
matrix consisting of the three airlines and the three responses "agree/neither/disagree". There 
results were not significantly different from those expected by chance, thus returns were 
combined across airlines for later analyses. 

Next, each of the 36 questions was separately analyzed for airline differences by constructing 
a 3 (airline) X 2 (agree/disagree) contingency table for each of the 36 questions. The "neither 
agree nor disagree" category was omitted because it typically is used to indicate an inability to 
respond to the question as well as a neutral feeling about the statement. The questions for 
which the pilots of different airlines gave similar answers and dissimilar answers are shown 
in Table 4. 'Die chi-square probability should be interpreted as the probability that the air- 
lines had identical responses to the questions. 

Captains Versus First Off-cers Each of the 36 questions was examined to determine if cap- 
tains and first officers responded differently. This was done by constructing a 2 (captam/FO) 
X 2 (agree/disagree) contingency table for each of the 36 statements. There were 11 state- 
ments in which the captains and first officers agreed (see Table 5), and there was significant 
disagreement on two statements: the captains agreed (and the FOs disagreed) that the autoland 
capability enhances safety, and that "automation frees me of much of the routine, mechanical 
parts of flying so 1 can concentrate more on managing the flight". 

Total Flying Time and 767 Flying Time An analysis was performed on the answers 
received to the 36 opinion questions to determine the correlation of total flying time, 767 
flying time, and captain/first officer differences with these answers. This was done by per- 
forming a discriminant analysis to see if the three variables could discriminate between the 
two categories (agree/disagree) on each question. While there was some effect for a few state- 
ments (e.g., 767 time predicted agreement with the statement "I can find the exact location of 
important controls and switches w'ithout any hesitation"), in general, the percentage of correct 
classifications of responses on the basis of these three variables was always less than 70%, so 
there seems to be almost no detectable relationship between the agree/disagree responses and 
the three variables. Note, however, a contingency table analysis did detect differences 
between captains and first officers on two of the 36 questions (see above). 

Factor Analysis The responses to the 36 questions were subjected to a factor analysis (there 
were 96 complete iesponses for this purpose). An examination of the percent variance 
explained versus the number of factors showed no significant "knee” in the curve, but that 8 
factors explained slightly more than 60% of the variance. These 8 factors were then rotated 
nonorthogonally to simplify the interpretation of the loading matrix; the factor loadings are 
shown in Table 6. This analysis is performed by the statistical program; it sorts the 36 ques- 
tions by the magnitude of the loading, if the loading is greater than O.f^and sets loadings of less 
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than 0.25 to zero. The questions below the dotted line in the table had the largest loading on 
that factor although the loading was between 0.25 and 0.50 in magnitude. 

It is encouraging to find that the factors are easily identified, but discouraging to find that a 
large number of factors are required to explain the variance. This seems to be another man- 
ifestation of the complexity of the human factors of automation. 


Interviews and Meetings 

In addition to the results previously reported under the open-ended questions, the following 
points emerged in discussions with the check pilots during interviews or meetings: 

1. Several of the pilots would have liked more training with the AFDS and Mode 
Control Panel. 

2. The first few trainees did not have a good grasp of the FMS/CDU when they 
reached line training, but this situation improved after each airline gained experi- 
ence. 

3. Two check a.rmen commented on the duties other than flying. Both felt that some 
crew members might get overloaded if these duties were not spread out in time. 
This was esj>ecialiy true when some equipment (e.g., ACARS) was not working. 
One check airman said that the two man crew should be given extra consideration 
(e.g„ different flight-plan forms, equipment to carry). 

4. Some crew members have had difficulty adapting to the two man crew concept. 
This seems to depend substantially on previous experience, e.g., narrow -body two- 
man experience versus wide-body three-man crews. One first officer said "there is 
nothing worse than a three-man-captain in this tw’o-raan airplane", indicating, as 
others did, that the captain must take an active role with extra duties when he is 
the pilot not flying. 

Cockpit Observation 

The experience of riding with crews on normal line trips was an extremely important part of 
the study and yielded information and insight that helped organize many facets of the study. 
Although no quantitative data were taken (as per the ground rules for the study), the follow- 
ing points were noted: 


1 . The pilots were extremely enthusiastic about the airplane, and took pride in its 
performance and the capabilities of the equipment. 

2. They were quite facile with the CD Li for some tasks, e.g.. building new waypoints 
for abeam fixes. Their performance in other CDU operations was more variable, 
such as setting up crossing fixes, and depended on their experience level. 

I 

3. On at least four of the segments we experienced the early-capture-at-low-altitude 
phenomenon reported by the pilots. In every case, both pilots were surprised by 
the sudden reduction in thrust as the aircraft leveled off, even though they had 
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selected a higher altitude by that time. 

4. There were many occasions when it was observed that the automatics were not 
performing the, task desired by the pilot. Most of these situations were minor 
discrepancies and were resolved by reprogramming the AI DS or FMS. A few of 
the pilots turned the autopilot completely off and hand-flew while re-engaging the 
systems. In at least six other instances, the pilot flying tried to rectify the situation 
by changing modes or setting new values in the AFDS or FMS, but these actions 
either did not immediately improve the situation or they made matters worse. In 
these cases the pilot seemed to become more uncertain of the true situation as he 
did more programming. 


Incident Reports 

Two incident reports were examined for the relevant human factors and automation elements. 

The first incident involved an unselected mode change from Flight Level Change (FLCH) to 
vertical speed with subsequent airspeed decay. The alerting and warning system remained 
silent until the appropriate angle of attack limits were reached. Then, as the pilot described it, 
"all hell broke loose" with the sudden onset of alerts. The pilot reported he was able to think 
of several reasons why the airspeed indicator was incorrect, even though it was correct. This 
follows a human tendency to retain the previous hypothesis during these first few seconds. 

Another crew member was involved in an in-flight spool-down of engines, resulting in tem- 
porary loss of the CRT displays. When the CRT displays were present, the F1CAS was filled 
with messages, and he had difficulty assimilating the information except for the only red mes- 
age (a cabin altitude warning). He could not discriminate between the second-level caution 
nessages (yellow, starting in the left margin) and the advisory messages (yellow, indented one 
.pace from the margin). Furthermore, he had many questions; "1 turned on the APU, is it 
»ming up to speed or not? Are the engines really running or are they windmilling 0 ’ He 
felt another crew member might have been useful, not as much for executing procedures as 
for helping diagnose the problem. 


- 20 - 


DISCUSSION 


Pilot Acceptance of the New Technology 

<? 

T he Airplane in General 

The pilots feel positively about the airplane. More than 86% agreed they "enjoy flying the 767 
more than the older aircraft" (#11). In response to a statement (#34) about the enjoyment of 
hand flying, one pilot remarked "It’s a sweetheart— tough to turn it over to automation!" This 
enthusiasm was also evident during the pilot interviews and the cockpit observations when 
the pilots also mentioned the aircraft performance (high clrmb rate and cruise altitudes) and 
the low fuel consumption. 

The New Cockpit Technology 

The pilots also seem accepting of the new cockpit technology, they choose to use it, and they 
find it helpful. Over 87% say they “like to use the new features of the 767 as much as possi- 
ble"(#18), 79% "use the automatic devices a lot because I find them useful"(#10), although 
31% also agreed to some degree that they "use automatic devices mainly because the company 
w’ants me to" (#35). 

The items mentioned by the pilots are shown in Table 2. Particularly noteworthy is that the 
general capabilities of the AFDS, FMS/CDU, and E1CAS are mentioned, suggesting their gen- 
eral agreement with the functions and implementations. Specifically mentioned items, such at 
the map display and autothrottle, are also heavily used as seen in the Frequency-of-Use table 
(in spite of their complaints about the implementation details of the autothrottle). 

Workload 

The pilot acceptance of the new cockpit technology, with respect to workload reduction, 
seems divided into two groups: those who say it reduces workload, and those who feel operat- 
ing the devices creates a form of workload. This is reflected in the divided responses to 
several questions: 47% ague and 36% disagree, that "Automation reduces overall workload" 
(#32); 53% agree and 37% disagree that "automation does not reduce overall workload, since 
there is more to keep watch over" #15; yet 79% agree that "1 use the automatic devices a lot 
because I find them useful"(#10), regardless of any workload penalty. A workload issue for 
which there was a significant difference between captains and first officers seems based on 
their different roles; captains agreed more, on the average, and first officers disagreed more, on 
the average, that “Automation frees me of much of the routine, mechanical parts of flying so I 
can concentrate more on ’managing’ the flight" (#24). 


Equipment Reliability 

Pilot opinion about the reliability of the equipment was measured by some of the attitude 
questions and roughly one-fourth of the pilots expressed some concern. Twenty percent of 
the pilots disagree with the statement "The new equipment is more reliable than the old" 
(#29) (45% agreed with the statement, and 35% neither agreed nor disagreed). Similarly, 27% 
agreed that they were "worried about sudden failures of the new devices like the FMS com- 
puter and the CRT displays" (#?), although the majority. 64%, disagreed with the statement; 
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and 26% agreed that they "have serious concerns about the reliability of this new equipment," 
and again the majority disagreed (62%). 

Skill Maintenance 

Maintenance of flying skills was a concern of the pilots. This appeared in the questionnaires 
and in the pilot interviews. For example, 87% agree that they “hand-flv part of every trip to 
keep mv skills up" (#14), and 80% agree that "pilots who overuse automation will see their 
flying skills suffer" (#18). Interestingly, this concern for other pilots did not always carry 
over to themselves because only 63% agreed that "1 am concerned about a possible loss of my 
flying skills with too much automation" (#31). It is felt, however, that some pilots did not 
agree with this statement because they do a lot of hand-flying. 

The Frequency-of-Use table shows that the pilots, in general, hand-fly during transition and 
en route climb (especially at the lower altitudes, as observed on line flights) and in the termi- 
nal area and final approach phases. 

Features Disliked 

There were few features or concepts that the pilots did not like, although there were features 
whose implementation, they felt, needed improvement. 

FMC Response Delay A large number of pilots felt that the response time for the Flight 
Management Computer was excessive. When a specific instance was mentioned, it usuailv 
involved complying with ATC requests while maneuvering in the terminal area. Although 
some of the pilots have learned that they can “type ahead" of the FMC, that is, push the 
appropriate buttons before the display requests the information, no pilot said he did this in the 
terminal area when rapid, accurate responses were required, perhaps because it has the poten- 
tial for committing errors. 

M echanical/ Electrical Checklists Two of the participating carriers used cardboard checklists, 
and one used a mechanical checklist. Pilots of the first two carriers felt some aid would be 
useful, especially as one pilot commented, it is difficult for a two man crew to get through a 
checklist w ithout some form of interruption. Many of the pilots felt that having the check- 
list displayed on the F.1CAS would be beneficial. Perhaps so, but previous experiments (Rouse 
and Rouse, 1980) have found that simply transferring material to the CRT does not neces- 
sarily improve performance. It should be noted that the presence of the air-start envelope 
parameters on the EICAS is not inconsistent with the concept of checklists on the CRT. 

Location of Circuit Breakers and Spare Bulbs Several pilots commented on the inability to 
reach circuit breakers and spare bulbs while remaining in their seat. This appears to be a 
result of having to pull circuit breakers frequently during the early months of line operation 
(to remove nuisance EICAS messages). The need to do this has been decreasing as system 
parameters are adjusted. 

Although the indicators have more than one bulb, one pilot reported having both bulbs in the 
landing gear indicator burned out. The cockpit design philosophy clashes with the reality of 
line operation at this point: should the pilot continue the landing without leaving his seat, or 
should he get up to replace the bulbs? Only more experience can answer this question. 
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Control Wheel Steering This autopilot mode was rarely used by the pilots, and some said its 
use was discouraged during training. Several reported in interviews that it was "rachety * and 
"abrupt." My experience with pilots of other wide-body aircraft, who also seem to ignore 
control wheel steering, suggests that there is more than rough performance behind this choice. 

From a human-factors view, CWS has the disadvantage that it alters the stimulus/response 
characteristics of the airplane, ia., control column movement to pitch-attitude response. This 
has the potential for causing "mode" errors, where control movements are generated for one 
mode, but the other mode is active. Discussion with pilots during the interviews revealed 
another aspect: CWS, the analog of the pitch/turn knob and/or vertical speed wheel of older 
autopilots with a "manual" mode, does not give them the appropriate control for certain 
phases of flight. In particular, it can be difficult to do the maneuvers the pilots wish, e.g.. 
Visual Meteorological Conditions (VMC) maneuvering in the terminal area. This seems to be a 
characteristic of all aircraft that have Mode Control Panels and CWS. Consider the following 
task: flying a VMC departure down a river that requires holding altitudes from time to time. 
This is a case where it is desired to regulate bank angle, instead of heading. This is easily done 
with a turn knob (which controls bank angle) and vertical speed wheel with altitude-hold 
detent; it is performed with one hand resting comfortably on the center console without look- 
ing at the controls. It is difficult to do the same maneuver with Mode Control Panels that 
have CWS; or heading select (this must be turned slowly to modulate bank angle) and a verti- 
cal speed wheel, two separate controls at arms’ length. 


Points of Confusion and Surprise 

This section will discuss the items reported by the pilots that relate to their operation of the 
Autopilot/Flight Director/ Autothrottle and the Flight Management System. The items 
reported by the pilots about the operation of the CDU will be deferred to the section under 
training. 

Autothroitle-V/S-SPD Interactions 

About 25% or the pilots reported experiencing some confusion, or seeing others become con- 
fused about the interaction of the autothrottles and autopilot. The source of this confusion 
seems to be twofold. 

First, the thrust/elevator combination is a complicated interaction in any aircraft, and it 
recalls the seemingly endless debate about controlling speed/altitude with throttle/elevator. 
Obviously, both strategics are possible in climb and descent. (There is agreement in some 
regimes, such as constant altitude: elevator controls altitude, thrust controls speed.) When 
these functions are automated, then, confusion and surprise are likely to follow if the pilots 
are not aware of the modes actually in use. The now-classic situation for the 767, reported by 
7% of the respondents but experienced by almost everyone, is the situation of a high climb 
rate close to the ground with a low altitude restriction. The autopilot "captures" the selected 
altitude about 1500 ft below that altitude, and switches from a mode where autothrottles are 
holding climb thrust and elevator is controlling airspeed, to a mode where the throttle con- 
trols airspeed and elevator controls altitude. It seems that just after altitude capture, an event 
that is not noticed by the pilot flying because he is also looking outside, an ATC clearance to a 
higher altitude is received. The new' altitude is selected, but instead of continuing the climb, 
the aircraft levels off at the "old" altitude and the throttles come back to maintain the previ- 
ously set, but not yet changed bug speed (about 170 knots). 

The second proposed reason for the confusion of the autopilot/autothrottle interactions, is that 
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this design has more capabilities than previous systems had. The autothrottle is almost always 
"armed"; in this state, it can become engaged, e.g., by engaging the SPD mode, even though it 
had been turned off with the throttle-mounted switches. Most pilots are used to autothrottles 
that can only be engaged by an autothrottle switch. The response to the questionnaires and 
experience in line observation suggests that there is some uncertainty about the conditions that 
will allow the autothrottles to become engaged. In addition, the throttles seem to come out of 
idle during descent at times that the pilots feel are inappropriate. 

Almost 10% of the pilots reported some discomfort with the speed synchronization at the time 
the Flight Level Change (FLCH) mode is engaged; FLCH is designed to climb at the existing 
IAS and climb thrust. The reason for the confusion seems to be that the SPD window shows 
a value at the time FLCH is engaged, but this value has no bearing on FLCH operation since 
the displayed speed automatically changes to the existing speed when FLCH is engaged. These 
pilots felt that FLCH should hold the speed displayed in the window, instead of the existing 
speed. Perhaps the confusion arises because the other numerical parameters on the mode con- 
trol panel (altitude, heading, even speed itself) operate as selected, not held, values. 

It is difficult, from the available data, to allocate the the autothrottle 'autopilot confusion 
among the several possible sources; system design, system implementation, training, and lack 
of experience with the aircraft. 

Al-'DS Turns " Wrong Way" or Does Noi Capture 

Nearly 20% of the pilots reported that at one time or another, the autopilot either turned the 
wrong way (usually on LOC intercept or when passing over a waypoint), or did not capture 
the desired route or course. It is impossible from the reports received to attribute these 
occurences to a lack of system knowledge, incorrect programming of the system, or equipment 
malfunction. Even if the pilots could be contacted for more information, it would be difficult 
for them to recall all the pertinent details, and in addition, they may not know what caused 
the anomaly. Some pilots, in their response to the question "Have you ever been surprised by 
the automatics" answered in the affirmative, but said they never had the time to determine 
why. 

One check airman suggested that an incorrect setting of the FRONT CRS knob on the Instru- 
ment Landing System (ILS) receiver w r ould cause the aircraft to turn the wrong way on LOC 
intercept; the aircraft will start turning to the incorrect course, but the ILS signals will even- 
tually cause the aircraft to track the localizer correctly. (One respondent mentioned he felt 
the ILS receiver was too far from the normal scan pattern, and so an incorrect setting might 
be missed.) There is also the possibility, mentioned by another check airman, that the appear- 
ance of the trend vector and the wind correction both contribute to a perception that the air- 
craft is turning away from the localizer w’hen it is not. 

Reports of turning toward the approach course before reaching it may possibly be attributed 
to capture of a localizer sidelobe. Reports of turning the wrong way after passing a waypoint 
are hard to explain, except as postulated by one pilot, that perhaps the autopilot had been fol- 
lowing the alternate route in the FMS. 

The causes of reported "failure" of the FMS to capture a course are difficult to determine. It is 
true that several preconditions must be satisfied before capture will occur, and it was noted 
that not everyone was aware of these preconditions during the early phases of operation. 
Still, equipment malfunctions or idiosyncrasies cannot be ruled out as contributors to the 
reported instances. 
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Use of the Wrong Control 

Pilots report using the wrong control knob, especially the heading knob for the speed select 
knob, and vice versa. This seems to occur during the first few hours on the airplane, and 
disappears with exposure; there were no occurrences observed on the line trips. 

Unselected Mode Changes 

This phenomenon was reported by 12% of the pilots, with all but two reporting a change to 
vertical speed, and the others reporting a change to heading hold; both are the default modes 
of the autopilot. One incident (to be discussed later) was precipitated by such a change. Most 
of these seem to have been of hardware, not operational, origin. 

The level off at FL.180 was a singular report. Before this report was received, an engineering 
pilot from a participating carrier noted that it might be possible to obtain such a level off if 
the altimeter set knob was turned fast enough and far enough (as might happen while passing 
through FL180) to cause the altimeter needle to move in the other direction. 

Training 

Introduction of a A'ew Aircraft 

The demands placed on a training department during the introduction of a new aircraft are 
great indeed. Their work starts long before certification, when the curriculum, slides, and 
tapes are designed with the manufacturer. From the time spent in ground school when the 
participating carriers were training their initial crews, it was obvious that their job was a 
difficult one: by necessity, much of the information they needed was not available, and the job 
of updating and inserting material is a never-ending one. This was compounded by the 
change from a three man to a two man cockpit only months before certification. In addition, 
the training staff must respond to the experience of their line instructors. 

Conversations with personnel involved in the transition training suggested that pilots felt the 
material fell naturally into three topics: aircraft systems, the Autopilot and Mode Control 
Panel, and the Flight Management System. In some sense, the same was true for the instruc- 
tors and program developers. Both the pilots and instructors seemed more at home with the 
aircraft systems, and these were learned without any appreciable difficulty even though they 
sometimes contained more automation than previous systems; e.g., electrical source selection. 

Some pilots and instructors had previous experience with mode control panels. Instructors 
felt strongly that this previous experience made the transition easier for pilots with this 
experience. 

The Flight Management System was entirely new to most instructors and pilots. Although 
some had prior experience with inertial navigation systems, the extensive capabi’ ties of the 
FMS, and its integrated nature were completely new to most individuals. The following com- 
ments, from the questionnaire, reflect this view. 


’[The FMS/CDU] system is complex and so completely different." • 

"I believe that the FMC was the most difficult to understand during ground school 
and the first few periods in the simulator. My classmates felt the same way." 
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FMS/CDU Training 


When asked on the questionnaire what materia] they wanted more or less of in training, the 
strongest responses were, requests for: more RMS and GDU training (in general); more "hands 
on" experience and training with the FMS/CDU; more line-oriented ('DU exercises; and less 
nonoperational CDU material. These comments were confirmed by several line training pilots, 
who, in the early phases, felt that the pilots arrived for line training with less than desirable 
knowledge and skills about the FMS/CDU. No pilot who responded to the questionnaire had 
training with a Flight Management Systems Trainer (FMST), although two of the airlines had 
ordered such a device. Most of the FMS/CDU exercises were done on part-task, computer- 
graphics terminals that illustrated the CDU keyboard and display. One airline attached a 
CDU keyboard and display to the instructional station; another developed a multi screen 
presentation to show, with slides, the mode control panel and the HAD1/EHSI; the FMS / CDU 
was depicted on an interactive computer graphics terminal. 

The difficulties of conducting the FMS/CDU training seemed to have come from several 
sources. First, there were many new concepts for the pilots to learn, e.g., navigating from 
autotuned radios, not from a single radio. Second, although it is beyond the scope of this study 
to identify the conceptually-’Sifficult aspects of the system, the organization of the informa- 
tion, and the naming conventions seemed to cause problems for some people. Third, and 
perhaps most important, there was no training device that (from the pilots' view) was an ade- 
quate simulation of the real FMS/CDU; see the comments below on Computer-Aided Instruc- 
tion. 

Relevance of Material 

It can be seen from the responses that many of the pilots wished they had had more "realis- 
tic" or line-oriented material in their FMS/CDU exercises, and /or less material on features 
that were nonoperational. This latter request seems to have arisen from the scheduled versus 
actual introduction of equipment capabilities. A’ first, the full capability FMS was to be 
introduced, and training reflected this. Subsequent schedule slippage resultec in the initial 
aircraft being delivered without a VN'AV (Vertical Navigation) capability; an "interim” 
VNAV package was later released, and the “full" VNAV capability is now scheduled for 
release in 1985. Thus, while training traditionally teaches some material before it is available, 
the actual evolution of the FMS made the VNAV material particularly irrelevant in the 
pilots’ view. 

!n addition to the material they received that they did not need, the pilots also felt that they 
did not receive material they could have used. In some sense this is a continuing point of con- 
tention between line pilots and training departments. In the case of the FMS/CDU, pilots 
revealed in interviews that they did not know how to deal with tasks such as crossing res- 
trictions until afte. they started line flying. Although one can argue that these functions 
would have been covered by the VNAV system, pilots were not given an interim method and 
sometimes did not receive the material in line training. Another item mentioned in the inter- 
views, and the questionnaires, was a last-minute change in approach assigned by ATC; remov- 
ing old information seemed to be as much of a problem as selecting the new approach from 
the menu. 

Computer-Aided Instruction— 

Impressions and Lessons Learned 

During the course of attending ground school at the three participating airlines, certain 
impressions were obtained from first-hand experience and the comments of classmates. These 
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impressions are being presented here, not to imply that one training method or training device 
is better than another since many other factors (e.g„ current staffing levels, staff capabilities, 
budgets, etc.) must be incorporated into a decision to use a particular training method or dev- 
ice. Rather, they are given in the context of experimental data so that consideration to these 
points can be given in the future. 

Realism A good deal of CDU instruction was done with computer-graphics terminals. This 
seems to be a reasonable teaching device for these tasks, since it is possible to create a good 
representation of the CDU on the graphics terminal and the touch-sensitive screen allows 
pilot actions similar to the real keyooard. The same computer system and programs were used 
by all three participating carriers and provided a basis for comparison. 

The pilots felt the primary drawback of the device was the lack of functional realism, i-e., at 
a given point of the exercise, there was usually only one allowed sequence of responses, 
whereas on the real system, much more freedom is available. The trainee was never sure 
whether the inability to do what he wanted to do was due to his lack of understanding or a 
limitation of the training device. It is recognized that this is a tradeoff involving program- 
ming effort, but the frustration level of the pilots became high at certain times. One carrier 
minimized this effect by having an instructor present with each crew as they went through 
the exercises. Before the pilot would start a sequence not allowed by the program, the 
instructor would ask his intentions; if they wr.e inconsistent with the computer program but 
consistent with the real system, the instruc.toi would say "Yes, you can do it that way on the 
airplane but this program is looking for another way, so do it this way,..." This approach, 
while requiring more instructional manpower, eliminated most of the frustration with the 
training device. 

In summary, it seems important to have the training device respond as much like the real 
device as possible without any artificial restrictions; this will remove the extra uncertainty in 
the pilot’s mind as he is learning, and will more quickly increase his knowledge of the system 
as he explores and makes mistakes using the system. Manuals for complex systems rarely tell 
what you cannot do. 

Self-Paced Instruction One carrier had most of the material on a computer instruction sys- 
tem tied to a mockup of the relevant panels, with backlighting controlled by the computer 
system. Conventional slides and audio tapes were also available to use as the pilots wished, 
ground school instructors were available to answer questions, and meetings with rated 
pilot/instructors were also used. The success rate on the oral exams was excellent. Originally, 
the pilots were instructed to proceed at their own pace through the computerized material 
over a period of more than a week, and were not given feedback on their pace. 

The pilots felt positively about the flexibility of scheduling their own time, but wanted feed- 
back on their progress. Most pilots felt they hurried through the multiple choice questions 
without much reflection. First, they did not know their progress compared to the norm, and 
they knew that a significant effort would be required if they got lehind. Second, without pac- 
ing of any kind (as might come from an audio tape) the NASA observer felt, as did others, 
that there was a tendency to "rush" through the questions, just to "finish." It seems that 
daily goals and some pacing (perhaps controlled by the student) would be useful. 

Prompting and Feedback No computer instruction that was used by the NASA observer 
allowed the student to control the level of prompting he received. At some point in the 
learning process, students reached the point where they felt they knew enough to try a solu- 
tion without any prompting or help, but they had no control over the prompting. 
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Prompting in computer-aided instruction is similar to a ’help" capability on interactive com- 
puter systems. However, with these systems the user is allowed to select the level of ’help" 
he receives, or he receives help only when he asks for it. Letting the user select the help level 
is a feature that has evolved alter many years of trying, without success, to “compute" the 
level of help needed from 1 the user’s previous experience and mistakes. User-selected help level 
is easier to program, and gives the user more control over the process; it would seem to be a 
useful feature to have in these instructional programs. 

The "three-screen" presentation developed by one airline was a useful step in integrating the 
many interactions of the FMS and autopilot systems. It was paced by an audio tape, and the 
NASA observer felt that one feature, in particular, was extremely useful as a teaching 
method. He knew enough to select the correct switch or knob to get through the exercise, but 
the audio tape and displays pointed out the many ramifications and concomitant system 
responses to those actions- This was extremely informative, and it would seem to be a useful 
goal of any training system dealing with interactive, integrated systems, since it reinforces the 
interactions and helps the student "build" Ins internal representation of the systems. 

Lectures vs. Computer- Aided Instruction The training results obtained by the three carriers, 
and the data from the questionnaires, do not suggest a superiority in transferring knowledge 
for the lecture format or the (primarily) computer-aided instruction. No pilot answering the 
questionnaire mentioned the positive aspects of flexible scheduling, although seveial students 
mentioned it during casual conversation. The 10 pilots who said they wanted less computer- 
aided instruction and more communication with instructors and classmates did not indicate 
whether they would give up schedule flexibility for these missing features. 

With the passage of time, it seems that the "pure" computer-aided instruction of one carrier 
has evolved to a program containing more interaction with the rated instructors. Moreover, 
the airline that traditionally used the lecture format has made extensive use of computer- 
aided instruction for the first time. Thus it seems that all airlines are evolving their training 
to a similar mix of computer-aided and face-to-face instruction. 

Is There an Information Cup? 

Observrtions in ground school, data from the questionnaires, and conversation with the system 
designers suggest that not all the desired information is making its way to the line pilots, at 
least during the introductory period. 

During the third ground school attended by the NASA observer, the instructor was describing 
the procedure to set up the FMS/CDU. He noted that it had been mysterious to him until he 
"discovered" that, once on the initial page, "if you press the LINE SELECT key on the lower 
right, it will lead you to the next logical step; if you want to back up. press the LINE SELECT 
key on the lower left". Each line key is labeled with a prompt for the appropriate "page" in 
the sequence. Prior to that time, the NASA observer had not remembered such a simple, logi- 
cal "rule" ui previous ground-school material, although it may have been there. It seems that 
the system designers had intended that this technique be used, but it w as apparently dropped 
somewhere along the way. 

Another example involves the use of the FMS'CDU in terminal area operations. A significant 
number of pilots reported on the questionnaire that they observed or experienced confusion in 
responding to rapidly changing ATC requests, and that the EMC responds too slowly under 
these conditions. Discussion with the cockpit design team revealed that they had anticipated 
the necessity for rapid response, and had incorporated features in the design that w'ould allows 
pilots to immediately respond to ATC and defer CDU interaction. No explicit instructions for 
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this situation were observed in ground school, simulator sessions, or line training flights, so it 
may be that the recommended procedures are not being transmitted to the pilots. Alterna- 
tively, the recommendations may may have been tried, but found inadequate. 

It is interesting to note that both cf these examples might be classified ;.s techniques, and tech- 
niques are of great interest to the pilots and the designer 1 . This is especially true when com- 
plex systems are involved because the right technique can greatly simplify n system, llut in 
addition to techniques, there is a large amount of detailed material to lie learned, and it is not 
surprising that techniques and other concepts get “lost" somewhere between the designers and 
the pilots. 

Computer Concepts 

Two of the questionnaire respondents askea for some instruction on computer concepts. 

"Ground school should not t-ach just function cf the CDU/computere, but a philo- 
sophy of computer applications and programming as applicable to our aircraft. This 
was done when the [new jet turbine technology] B-707 was introduced in 1958. 
Now that everyone is jet oriented, this is not necessary. So today, the computer is 
new and should be taught until everyone has the 'idea’". 

"For these of us with no computer literacy (buzz word) a 10 minute dissertation 
on computer functioning would help. Actually, just the thought that the damn 
thing only does what it is told would save some errors." 

Gr“ pilot suggested an even broader scope. 


"From what I’ve seen so far, we could use a bit more emphasis on the ’background’ 
of some of the automatics to better able a crew to understand what’s happening or 
not happening when things don’t go as programmed..." 


This t 5 T pe of instruction would certainly be consistent with the idea of creating a "schema" or 
framework about computers or automation, into which detailed information would more 
easily be assimilated. 


Flying The New Technology 

Distractions 

Several incident reports have appeared in the last 2 yrs that have a common theme. The 
incidents involved transport aircraft with higher levels of automation. A typical scenario 
proceeds as follows: the automatics are on and doing their assigned tasks mo*e than ade- 
quately. Something happens to attract the crew members’ attention; e.g., crossing a navigation 
fix, distractions from other crew members, etc. These distractions are typified by high levels 
of cognitive (not peieeptual) activity. During this time, something happens to the operation of 
the automatics (unselected mode change, unusual environmental conditions) that requires 
intervention by the crew. This need goes unnoticed because of the human tendency to deal 
with one task at a ume, and the high cognitive levei of the “distractor" task that consumes 
most (if not all) of the crews' attention. 
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In some respects, this is exactly the tendency that must be overcome for a pilot to become 
proficient at instrument flying. Instrument fiymg involves "learning" (l) to extract the neces- 
sary information from a display as quickly as possible, and (2) not to fixate on one instrument, 
but to sample all instruments with appropriate frequency. Instrument flying breaks down 
when the pilot becomes "locked on" to an important piece of information and concentrates on 
resolving one anomaly to the exclusion of the other parts of the task (e.g- neglecting airspeed 
control during a difficult localizer intercept). 

The experience with all aircraft having more sophisticated on-board devices leads us to suggest 
that pilots must learn not to neglect the basic aircraft parameters. In other words, new scan 
patterns must be developed, so that the pilot can deal with "distractions" as well as with 
monitoring. During instrument flying, the pilot receives immediate feedback if he fails to 
properly scan the instruments, since the aircraft will quickly deviate from the intended 
course. Th pilot receives feedback about improper monitoring, however, only when the 
automatics fail to operate as intended and when he is otherwise distracted. These two condi- 
tions do not oft"n occur at the same time, and so feedback on improper monitoring is rare. 

Turn it of!! 

On several of the line observation trips, the NASA observer noticed the following. When 
things did not go as planned, or when the pilot was "surprised" by the automatics (e.g, the 
early altitude capture with high rate of climb), the pilot would try to "program" his way out 
of the anomalous condition. The situation would sometimes get worse and more confusing, not 
better. It seemed to the observer, on these occasions, that the pilot would have made a 
smoother and less distracting recovery by simply turning off all the automatics and then 
turning them on one at a time as needed. This strategy has the advantage that one immedi- 
ately starts f rom a known condition, a hand flown airplane, and it is much easier to assess the 
automatics as they are engaged one at a time. 

A captain involved in an incident spontaneously volunteered tha' his experience would not 
have degraded to the level of an incident had he turned everything off when he became con- 
fused and started "fresh". 

The tendency for some pilots to program a recovery, and not "turn it off." was also confirmed 
by interviews and discussions with line training pilots and check airmen. It does not appear 
to be a fascination with the new equipment, instead, it appears to be a habit learned during 
simulator training and most line training, where the instructor’s job is to ensure that the stu- 
dent learns the operation of the automatic equipment. It seems to be taken for granted that 
the student knows there is an airplane behind the panel, and that the student knows when to 
turn it all off. 

A questionnaire respondent felt the need for this training when he asked for more training 
"to turn off the auto system and take over manually at any place or time." 

One line-training captain said he used the following metaphor for new captains (it lias some 
disadvantages when applied to first officers!): think of the automatics as a crew member brand 
new to the airplane: take over from the automatics (i-e., turn it off) any fme you would take 
over from the new crew member. 
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In summary, it appears that pilots need "turn it off" training because of the tendency (perhaps 
due to prior training) to program their way out of an anomaly, but this often makes matters 
worse. 


o 
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HUMAN FACTORS PRINCIPLES: 
CONFIRMATIONS, A NEW PRINCIPLE, AND RESEARCH ISSUES 


This section describes the information gleaned from the study that has a bearing on the more 
general human factors issues. These have surfaced as confirmations of existing principles, a 
suggestion for a new principle, and topics for new research. 


Confirmations 


Minimize Mental Operations 

~> 

It has been proposed that displays be designed to minimize mental operations or transforma- 
tions (National Research Council, 1982). This principle seems to have been confirmed by 
several examples of the B-767 aircraft. First, the map display gives an excellent representa- 
tion of the horizontal situation, with minimal effort. It is well received by the pilots and 
heavily used, testimony in itself. (The ease of use and acceptance come as no surprise to the 
military pilots and researchers who have been using map displays for more than 15 years.) 

Another example of a display that minimizes mental operation is the altitude arc. This shows, 
on the map display, the geographical position at which the aircraft will reach the altitude 
selected on the mode control panel. This display eliminates the necessity to continually extra- 
polate the flight path, by rules of thumb, to determine the position where the aircraft will be 
at the desired altitude, i.e, will the crossing restriction be met, or will the descent be too steep 
resulting in wasteful use of fuel at low altitudes. 

Human Error 

As presented in an earlier work (Wiener and Curry, 1980). one goal of automation is to elim- 
inate human error. It was our contention that this is difficult, and automation will change 
only the locus and type of human error. This principle has not been disproved by observation 
of automation on the B-767, since the operation of this aircraft has not been without incident 
where human error was a contributing factor. There certainly are not enough data, nor is it a 
meaningful exercise, to determine if it is "better" or "worse" than other aircraft from this 
standpoint. 

The accident involving the Air New Zealand DC- 10 in the Antarctic, in which ground-based 
computer errors had been made while creating and distributing flight plans, may be a precur- 
sor of this change of locus of human error. There is scant evidence that such errors are occur- 
ring with the current database or systems on the B-767. There was only one reported case of 
an anomaly that apparently involved a stored flight plan taking a 50-mile off-track excursion 
to an obscure waypoint halfway through the cross-country flight plan. 
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A New Human Factors Principle for Automated Systems 

This section presents some information that appears to be worthy of consideration as a human 
factors principle of automation. 

Display Data, not Commands, for Control 

There are three examples that suggest that displaying data, not commands, is more useful if 
the pilot can take appropriate control action without significant mental transformations oi 
effort. The first example is the Heads-Up Display (HUD) format proposed by Bray (1980), 
where inertial flightpath angle and scaled ILS raw data can be combined to allow easy ILS 
tracking. The second example is the track-up map display and trend vector on the 767 (func- 
tionally similar to the HUD format). The third example is the altitude arc discussed above. In 
each case vehicle state information can be used to generate corrective .control actions w’ith lit- 
tle or no mental effort. Moreover, there is a great deal more flexibility in accomplishing flight- 
path objectives w'ith the data display compared to the command display. 

The issue seems to be this: A command-generation device needs to know the objectives of the 
maneuver. These alw'ays have to be transmitted to the pilot or to the command generation 
system, either implicitly or explicitly. A data presentation, on the other hand, needs no such 
ransmission of goals, and it allows the pilot the flexibility to use more knowledge than w'ould 
be feasible to incorporate into the command generator. Consider the following fictitious exam- 
ple of how a pilot could use the altitude arc on the B-767 map display: As they prepare to des- 
cend into O’Hare, the pilot thinks, "1 know ATC is going to clear me to cross XXX at 11, (XX), 
but it is Friday afternoon, the traffic is heavy, and 1 will probably get vectors off course for 
spacing. If 1 keep it a little high, 1 will not get down too early even with the vectors; the 
altitude arc will tell me how 1 am doing with the crossing restriction when I am off course. 
If ATC doesn't give me vectors, 1 can push it over and know that I’ll make the crossing res- 
triction without any trouble." 

Thus, disolaying data (not commands) for control removes the necessity for the pilot and com- 
mand gi ,-erator to transmit and/or have common objectives, and it allows the pilot flexibility 
to modify nis goals for existing conditions. 


Research Issues 

Display of Mode Target Values 

It is not unusual to see a pilot "fighting" the automatics, especially autothrottles: he pulls 
them back, they advance, he pulls them back again, and they advance once more. Obviously 
there is a goal conflict. In a system with many complex interacting modes, it seems plausible 
that explicit display of the goals or target values, as w’ell as the modes, would be an aid in 
interpreting the actions of the automatics. In an aircraft context, such displays might take the 
following form: 


- 34 - 


with target values 


without target values 


THRTL-250KTS 
ELEV"! 0,000 FT 


SPD 

ALT HLD 


or 


THRTL-CLIMB EPR 
ELEV-V/S-1500FPM 


EPR 

V/S 


It is true that aircraft autopilot and flight management systems usually display the target 
values, but such data exist in a variety of places and not necessarily near the mode annuncia- 
tions; moreover, mode annunciations do not always indicate who is doing what to whom. The 
questionnaire responses suggest that pilots view the central location of mode annunciation on 
the EADI as a positive feature; displaying the target values, along with the modes, has the 
potential for further enhancing a rapid determination of system state. 

Human Behavior and Information Processing for 
Partial Automation and Monitoring 

It was described earlier how pilots, as humans, tend to become distracted from the monitoring 
task. The possibility of more and/or different training was raised as a possible remedy, 
although it seems difficult to achieve consistent gains this way. It is recommended that the 
influence of system design be investigated as an alternative method to alleviate the problem. 
For example, lower-level advisory messages such as legrading airspeed, rising engine, or cargo 
temperature, may improve monitoring performance. The next section discusses these possibili- 
ties. 

The human information processing requirements are different for manual flying and flying 
with automation, and a better understanding of the differences would be useful to designers. 
For example, altitude is continually scanned when pilots manually fly the airplane; this in 
turn, may serve as a reminder for the level-off altitude. There is no such persistent require- 
ment when flying with automation, but one can argue that automation frees human resources 
to allow better monitoring. If this is so, why have there been so many spectacular lapses of 
monitoring in over the years? Pilots feel that flying with partial automation is different, and 
requires more monitoring. In one sense, it should be no different from monitoring another 
crewmember as he flies, but pilots do not think of it that way. Why is this so? Research is 
required to unravel these paradoxes. 

The time-dependent aspects of ronitoring also should be investigated. One may view flying as 
the performance of many procedures that are triggered by elapsed time and events. The intro- 
duction of automation introduces new events and alters the elapised time of familiar events. It 
is r>'en more difficult when the operators are not exactly sure what events may be triggered, 
and by what. Moreover, the absence of an event is usually more difficult to detect than the 
presence of an event, so detecting malfunctions from these symptoms is more difficult. System 
designers need to know more about the useful time intervals of monitoring and other related 
phenomena. 

Alerting Systems 

Direct informa tian was obtained from crew members involved in two incidents. Both discus- 
sions revealed a need for more information about human interaction with alerting and warn- 
ing systems. 
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Trending The incident involving airspeed decay reveals the problematic nature of w ithhold- 
ing all alerts until absolute necessary. Other incidents have also demonstrated this. It is 
human nature to retain the previous hypothesis in spite of alarm indications to the contrary 
(alarm systems are not always correct, after all). In many instances it would seem that the 
time it takes to come to a correct diagnosis can, and has been, lengthened by this alerting phi- 
losophy. A 'trending* philosophy would allow an assimilation of information over time, and 
may even prevent the situation from deteriorating to the point of normal alarm activation. 
The drawback of a trending philosophy, of course, is the possible existence of too many nui- 
sance alerts. Thus, the research should explore trending philosophy and determine how to 
display a large number of low-level advisory messages without being bothersome. 

Perhaps the design objective of any alerting system (as told by one pilot) should be this: if the 
pilot mentally says "thanks' to the system, it was a useful alert; otherwise it is not. 

Quantity and Quality of Information The report of the crew member who experienced 
engine spool-down raises the extremely difficult issues facing the system designers: what 
information should be given to the pilot; how should it be presented; how should it be priori- 
tized? 

At least three research topics are suggested by this incident, although the first is not new. Is 
there a logical and rational method, from both the designers’ and pilots’ view, to organize and 
prioritize the information? Second, is it worthwhile to have a sophisticated on-board system to 
propose hypotheses and actions to the crew? Lastly, would it be useful to have a database 
query system to answer questions such at posed by this pilot, e.g, is the APU in a normal 
start, or are the engines running? The interface to such a quepv system would be a design 
challenge, indeed, but a query system might remove the primary need for prioritization and 
organization of information. 
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CONCLUSIONS 


There are mo points concerning the results that deserve' particular emphasis. First, the data 
were taken during the early introduction of the aircraft and the conclusions apply only to 
that period. Second, although the B-767 was the only aircraft in the study, discussions with 
operators of the A310 (another new-technoiogy-cockpit aircraft) have confirmed very similar 
experiences. Thus, the following conclusions, while specifically mentioning the B-767, are 
likely to be valid for the introductory period of the A310: 

1. Most of the pilots enjoy flying the B-767 more than they enjoy flying the older air- 
planes. 

2. The pilots accept the ntw cockpit technology, and they choose to use it because 
they find it useful. 

3. The pilots are aware of the possible loss of flying skill with the presence of auto- 
mation, and they hand-fly (usually with flight director) to prevent this loss. The 
data collected in this study do not indicate any loss of skills. 

4. The primary points of confusion or surprise were autothrottle/autopilot interac- 
tions; the autopilot turning the ’wrong way" or not capturing the course; and 
achieving desired results with the (FMS/CDU). 

5. The pilots felt training for the FMS/CDU could be improved, and they especially 

wanted more "hands on’ experience. More training on the mode control panel, and 
more hand flying were also mentioned. J 

6. Information, especially "techniques," may not always be getting from the system 
designers to the line pilots. 

7. Flying any aircraft with sophisticated equipment and high levels of automation 
allows distractions that cause a loss of monitoring performance. 

8. Many pilots should be trained to "turn it off" and not try to "program" their way 
out of an anomalous situation. 

9. These ueld oata confirm some existing human factors principles, suggest a new 
principle, and raise questions requiring further research. 
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Table 1. Pilot Statistics 


Airline 

Number of 
3f Captains 

Number of 
of I VOs 

Total Time (hrsl 

0 

767 Time Chrs) 




Minimum 

Median 

Maximum 

Minimum 

Median 

Maximum 

A 

IS 

7 

8000.0 

14000.0 

23150.0 

17.0 

60.0 

300.0 

B 

16 

12 

8500.0 

12000.0 

24000.0 

20.0 

113.0 

300.0 

C 

30 

22 

4200.0 

1 5500.0 

25000.0 

5.0 

103.5 

250.0 

411 Pilots 

61 

41 

4200.0 

1 3500.0 

25000.0 

5.0 

100.0 

300.0 
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Table 2. Number of pilots mentioning items 
on 102 questionnaires 


Total 

Number of 
Questionnaires 
102 

Number of 
Pilots 

FEATURES LIKED 

AFDS 

20 Autothrottle Concept/Speed Control 

14 AFDS Capabilities 

10 Takeoff Mode and/or EEC 

8 Reduced Workload 

6 Altitude Capture/Select 

EF1S 

42 Display and clarity of information 

22 Map display 

7 Green Altitude Arc 

5 Wind Vector 

4 ADI Mode Annunciation 

2 Ground speed display 

FMS/CDU 

48 System capabilities 

2 Route display 

EICAS 

35 Quality and quantity of information 

6 Engine limits and numbers 

3 Monitoring capabilities 


FEATURES MISSING OR NOT LIKED 

20 FMC response delay 

7 Want electrical/mechanical checklists 

7 Circuit breakers and spare bulbs not within reach 
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Table 2. Number of pilots mentioning items 
on 102 questionnaires (con) 


Total 

Number of 
Questionnaires 
102 

Number of 
Pilots 

POINTS OF CONFUSION OR "SURPRISE" 

25 Autothrottle-V/S-SPD Interaction 

20 AFDS turns "wrong way" or does not engage 
19 Using wrong control (especially HDG/SPD) 

12 Unselected mode change (10 to V/S, 2 to HDG HLD) 

11 Removing route discontinuities and extra information 

11 Track/heading on map display 

9 Speed sync at FLCH engagement 

7 Early altitude capture at high climb rate 
7 AFDS-MCP mode (general) 

6 FMS/CDU useage (general) 

6 Simultaneous speed brakes and landing flaps 
5 Changing approaches on FMS/CDU close-in 

3 No aural trim indication 

3 Holding with FMS/CDU 

3 Map drift 

2 Use of J routes in FMS/CDU 

2 High bank angles at LOC capture 

2 Defining waypoints from station 

1 Unselected level-olf at FL180 


TRAINING 

4 Satisfactory as is 

More: 

25 FMS/CDU 

22 "Hands on" CDU experience 

12 Hand flying 

8 AFDS-MCP training 

7 Practical, line-oriented CDU exercises 

6 Aircraft systems 

3 Single engine simulator experience 

Less: 

10 Computer aided instruction 

7 Three-man simulator 

3 nonoperational FMS material 

2 Phase-of-flight presentation 
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Table 3. Statements Used for Pilots’ Responses 


1. 1 can fly the airplane as smoothly and safely by hand as with automation. 

2. Younger pilots catch on to automation faster than older ones. 

3. Flying today is more challenging than ever. 

4. The FMS/CDU is easy to use in normal line flying. 

5. 1 think they’ve gone too far with automation. 

6. Autoland capability definitely enhances safety. 

7. I spend more time setting up and managing the automatics (such as the FMS/CDU) than 
I would hand-flving or using the old style autopilots. 

8. 1 like to use the new features of the 767 as much as possible. 

9. I am worried about sudden failures of the new devices like the FivlS Computer and the 
CRT displays. 

10. 1 use automatic devices a lot because 1 find them useful. 

11. 1 enjoy flying the 767 more than the older aircraft. 

12. I always know what mode the Autopilot/Flight Director Is in. 

13. 1 can fly as efficiently as the FMS without its help. 

14. I hand-fly part of every trip to keep my skills up. 

15. Automation does not reduce workload, since there is more to keep watch over. 

16. 1 can find the exact location of important controls and switches without any hesitation. 

17. Automation is the thing that is going to turn my company around and make it 
profitable again. 

18. Pilots who overuse automation will see their skills suffer. 

19. The ADI and EHSI displays are always legible and easy to read. 

20. 1 am favorable toward automation in the cockpit - the more the better. 

21. Flying the 767 is definitely easier than flying other aircraft. 

22. Setting piloting priorities with this new cockpit technology is no more difficult than in 
our other airplanes. 

23. We should have full autothrottles on all the company’s aircraft. 

24. Automation frees me of much of the routine, mechanical parts of flying so 1 can con- 
centrate more on "managing" the flight. 

25. I have serous concerns about the reliability of this new equipment 

26. Sometimes what the automatics do or don’t do takes me by surprise 

27. It is easier to cross-check the other pilot in the 767 than in our other airplanes. 

28. Too much automation can be dangerous. 

29. The new equipment is more reliable than the old. 

30. It is important to me to fly the most modern plane in the company’s fleet. 

31. I am concerned about a passible loss of my flying skills with too much automation. 

32. Automation reduces overall workload. 

33. 1 always feel I am ahead of the airplane. 

34. Hand-flying is the part of the trip 1 enjoy most 

35. 1 use automatic devices mainly because the company wants me to. 

36. The FMS/CDU requires little or no in-flight button-pushing below FL180. 
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Table 3. Pilot Opinion Summary (% responses by category) (con) 


STATEMENT 

STRONGL V 

SLIGHTLY 

NElThEit AGREE 

SLIGHTLY 

STRONG1 Y 

NUMBER 

AGREE 

AGREE 

NOR D-SXGREE 

DISAGREE 

DISAGREE 

1 

28 

31 

12 

25 

5 

2 

12 

37 

2S 

18 

6 

3 

37 

29 

14 

17 

3 

4 

38 

35 

5 

19 

3 

5 

3 

17 

IS 

27 

35 

6 

26 

36 

17 

15 

6 

7 

30 

35 


is 

14 

8 

54 

36 

«• 

4 

0 

9 

11 

16 

10 

CO 

36 

10 

39 

40 

let 

6 

0 

11 

62 

24 

- 

7 

0 

12 

29 

32 

S 

28 

3 

13 

3 

17 

IS 

40 

23 

14 

63 

24 

4 

7 

2 

15 

22 

31 

10 

23 

1 A 

16 

29 

29 

10 

31 


17 

6 

15 

39 

18 

21 

18 

48 

32 

t> 

12 

3 

19 

51 

28 

5 

13 

3 

20 

15 

44 

17 

17 

6 

21 

13 

33 

24 

25 

6 

22 

19 

32 

10 

32 

8 

23 

16 

21 

41 

15 

8 

24 

19 

42 

16 

19 

5 

25 

4 

22 

13 

30 

32 

26 

10 

52 

S 

22 

8 

27 

11 

26 

30 

27 

7 

28 

11 

34 

29 

17 

10 

29 

13 

32 

35 

21 

0 

30 

16 

28 

33 

16 

8 

31 

24 

39 

s 

16 

13 

32 

18 

29 

17 

31 

5 

33 

21 

40 

10 

28 

1 

34 

22 

38 

25 

10 

6 

35 

1 6 

25 

27 

30 

13 

36 

1 I 

13 

6 

24 

55 
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Table 4. Contingency Table Comparisons of Responses to Statements 
(agree/disagree) and Airlines 


Statements on which there was agreement (p >0.72) 

« 


Probabilitv 

Statement number 

Statement 

0.93 

2 

Younger pilots catch on to automation faster than old- 
er ones. 

0.95 

5 

1 think they’ve gone too far with automation. 

0.82 

11 

I enjoy flying the 767 more than the older aircraft. 

0.80 

13 

. 

1 can fly as efficiently as the FMS without its help. 

0.72 

20 

1 am favorable toward automation in the cockpit-the 
more the better. 


Statements on which there was disagreement (p <0.05) 


0.013 

1 

1 

1 can fl v the airplane as smoothly and safely by hand 
as with automation. 

0.015 

4 

The FMS/CDU is easy to use in normal line flying. 

0.037 

9 

1 

I am worried about sudden failures of the new devices 
like the FMS Computer and the CRT displays. 

0.048 

14 

I hand-flv part of every trip to keep my skills up. 

0.025 

16 

1 can find the exact location of important controls and 
switches without any hesitation. 

0.017 

21 

Flying the 767 is definitely easier than flying other 
aircraf t. 

0.041 

26 

Sometimes what the automatics do or don’t do takes 
me by surprise. 

0.025 

29 

The new equipment is more reliable than the old. 
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Table 5. Contingency Table Comparisons of Captains Versus 
First Officers and their Response to the 36 Statements 

Statements on which there was agreement (p >0.80) 


Probability 

Statement 

number 

Statement 

0.85 

1 

1 can fly the airplane as smoothly and safely by hand 
as with automation. 

0.S9 

4 

The FMS/CDU is easy to use in normal line flying. 

0.87 

10 

1 use automatic devices a lot because 1 find them use- 
ful. 

0.88 

12 

1 always know what mode the Autopilot/Flight 
Director is in. 

0.85 

13 

1 can fly as efficiently as the FMS without its help. 

0.91 

19 

The .ADI and EHSi displays are always legible and 
easy to read. 

0.90 

21 

Flying the 767 is definitely easier than flying other 
aircraft. 

1.00 

22 

Setting piloting priorities with this new cockpit tech- 
nology is no more difficult than in our other airplanes. 

0.87 

23 

We should have full autothrottles on all the 
company's aircraft. 

0.85 

32 

Automation reduces overall workload. 

0.84 

34 

Hand-flying is the part of the trip 1 enjoy most. 
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Table 5. Contingency Table Comparisons of Captains Versus 

First Officers and their Response to the 36 Statements (con) 

Statements on which there was disagreement (p <0.05) 


Piobability 

Statement 

number 

Statement 

Reasons 

0.047 

6 

Autoland capability 
definitely enhances safety. 

Captains agree more, FOs 
disagree more 

0.043 

24 

Automation frees me of 
much of the routine, 
mechanical parts of flying 
so I can concentrate more 
on "managing" the flight. 

Captains agree more, FOs 
disagree more 
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Table 6. Factor Analysis of Pilot’s Responses to 36 Statements 


Factor Loading 

Statement number 

Statement 

0.815 

25 

I have serious concerns about the reliability of this 
new equipment. 

0.806 

9 

1 am worried about sudden failures of the new devices 
like the FMS Computer and the CRT displays. 

-0.661 

29 

The new equipment is more reliable than the old. 

-0.525 

22 

Setting piloting priorities with this new cockpit tech- 
nology is no more difficult than in our other airplanes. 

0.732 

13 

1 can fly as efficiently as the FMS without its heip. 

0.727 

1 

1 can fly the airplane as smoothly and safely by hand 
as with automation. 

-0.534 

24 

Automation f rees me of much of the routine, mechan- 
ical parts of flying so 1 can concentrate more on 
“managing" the flight. 

0.462 

35 

1 use automatic devices mainly because the company 
wants me to. 

0.458 

7 

1 spend more time setting up and managing the au- 
tomatics (such as the FMS/CDU) than I would hand- 
flying or using the old style autopilots. 

-0.447 

32 

Automation reduces overall workload. 

0.409 

34 

Hand-flying is the part of the trip I enjoy most. 

0.327 

5 

1 think they’ve pone too fat with automation. 
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Table 6. Factor Analysis of Pilot's Responses to 36 Statements (con) 


Factor Ijoadine 

Statement number 

Statement 

0.802 

16 

1 can find the exact location of important controls and 
switches without any hesitation. 

0.768 

12 

1 always know what mode the Autopilot/Flight 
Director is in. 

0.727 

33 

I always feel 1 am ahead of the airplane. 

0.497 

10 

1 use automatic devices a lot because I find them use- 
ful. 

0.488 

4 

The FMS/CDU is easv to use in normal line fivinp. 

0.804 

14 

I hand-fly part of every trip to keep my skills up. 

0.753 

18 

Pilots who overuse automation will see their flying 
skills suffer. 

0.720 

31 

1 am concerned about a possible loss of my flying skills 
with too much automation. 

0.407 

28 

Too much automation can be dancerous. 

0.729 

30 

It is important to me to fly the most modern plane in 
the company’s fleet. 

0.716 

11 

I enjoy flying the 767 more than the older aircraft. 

0.549 

20 

I am favorable toward automation in the cockpit - the 
more the better. 

0.489 

8 

1 like to use the new features of the 767 as much as 
possible. 

0.458 

6 

Autoland capability definitely enhances safety. 

0.799 

36 

The FMS/CDU requires little or no in-flight button- 
pushing below FL1S0. 

0.648 

27 

It is easier to cross-check the other pilot in the 767 
than in our other airplanes. 

-0.379 

15 

Automation does not reduce workload, since there is 
more to keep W'atch ever. 
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Table 6. Factor Analysis of Pilot’s Responses to 36 Statements (con) 


Factor Loading 

^Statement number 

Statement 

0.701 

o 

23 

We should have full autothrottles on all the 
company’s aircraft. 

0.515 

21 

Flying the 767 is definitely easier than flying other 
aircraft. 

-0.451 

26 

Sometimes what the automatics do or don’t do takes 
me by surprise. 

0.448 

17 

Automation is the thing that is going to turn my com- 
pany around and make it profitable apain. 

0.645 

19 

The ADI and EHS1 displays are always legible and 
easy to read. 

0.548 

2 

Younger pilots catch on to automation faster than old- 
er ones. 

0.496 

3 

Flying tcdav is more challenging than ever. 
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Appendix A 


o 

Questionnaire Distributed to 
Participating Pilots 


PRECEDING PAGE w\NK NOT FILMED 
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Please fill In iho Information below, and than p»'c*ce**d with the remaining parts. 


When you are finished, return the entire booklet In the envelope to: 

Dr. Ren Curry 
NASA Ames, 239-3 
FWioffott Field, CA 94035 


ID code 


Your Position : Captain First Off icer 

Today's Date : / / 

Date you finished 

line training: / / 

Total hours fiying time : 

Total hours In 767 : 

Days 3lnce you last 

flow tha 7 67 : 
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FREQUENCY OF USE 


o 

The purpose of this part is to determine how frequently you use certain features 
of the 767 during various phases of flight. 

Enter the percent of your legs on '.jhich you use the particular featuie for each 
phase of flight. For example, if you always use autothrottle on takeoff, then 
enter 1 00 in the row for Autothrottle and the Takeoff column; if you use 
autothrottle for takeoff only on 1/3 of the legs, then enter 33 in the Takeoff 
column. 

If it is possible to use a feature In c particular phase of flight, but you never use 
it because of company procedures, F ARs, or your own preference, then enter 0. 

If it is impossible or doesn't make sense to use a feature during a particular 
phase of flight, e.g., "Step Ciimb" during the takeoff phase, then cross out tl at 
"cell" of the table with a large X. 

The last column is provided for any comments you may have about why you do 
or do not use the feature, and/or how you use it. 
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ID Code 



i 


For each phase-of-fii^rt colimn in ths table, enter the percent of you legs on which you use Uie leatue 
described on the left. If it Is impossible or doesn't make sense to use the feat ire in a partioier phase of 
flight, then cross out that "ceil" in the table. 


FEATURE 

PHASE OF FLIGHT 

COMMENT 

Takeoff 
to 10O0 
AGL 

Transition 
St Enrouta 
Climb 

Cruise 

Descent 
(Cruse to 
10,000 
MSL) 

Terminal 

Area 

Final Ap- 
proach 

Landng 

AUTOPILOT 

CMD 









CWS 







J 

Hand 

Fly 

no FD 









with FD 









MCP FEATURES 
Vertical Speed 









Bank limit— Auto 









Berk limit— man 








! 

LNAV 









VNAV 









FLCH 









Approach Mode 









Autoland 









AUTOTHROTTLE 








1 : 

DISPLAYS 
Map mode 









VOR/LS mode 








1 

Altitude (Green) Arc 









FMS/COU 

Direct/Intercept 









Step Climb 








| 

FIX mode 








1 

1 

VOR manual tine 
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ID Code 


For each pbase-of-filj^it column In the table, enter the percent of yoia- legs on which you use the feature 
described on the left. If Is Impossible or doesn't make sense to use the feature In a particular phase of 
flight, then cross out that "ceil" in the table 
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ID Code 


c GENERAL QUESTIONS 

1. Have you seen any confusion or incorrect operation on the part of other 
crew members in the use of these 767 systems? If so, what? 

AFDS (MCP) and 
Autothrottle 

ADI/EHSI 

FMS/CDU 


EICAS 


2. Have you ever been "surprised ’ by the actions of the automatics, that is, 
they did or did not do something you expected? If so, please explain. 


3. Are there any features about the systems below that you are not quite sure 
about, or that you do not feel comfortable with? Please descibe. 

Feature rot What do you think 

siue of the problem is? 


AFDS (MCP) and 
Autothrottle 

ADI/EHSI 


FMS/CDU 

EICAS 
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4. What feature or capability do you like most and like least about each of the 
following systems? 


AFDS (MCP) and 
Autothrottle 


ADi/EHSI 


FMS/CDU 


EICAS 


5. If you could make any changes in the cockpit (layout, add or delete features, 
lighting, ventilation, noise, etc.) what would they be? 


6. What area(s) should receive more or less emphasis in training? (Consider 
both ground and simulator.) 


Feature liked 
the most 


Feature liked 
the least 
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0 


767 PILOT QUESTIONNAIRE 


ID cods 


The following statements describe reactions to flying, new cockpit technology, 
and the 767 equipment. For each item, indicate how much you agree or disagree 
with the statements, as they refer to yourself, by circling the appropriate letter 
on each scale. 

ANSWER QUICKLY: YOUR FIRST IMPRESSION IS THE BEST. Remember to 
answer every question even if you are unsure. 

Feel free to add any comments after you have circled all the answers. 


1. I can fly the airplane as smoothly and safely by hand as with automation. 


A 

B 

C 

D 

E 

strorrfy 

siightiy 

neither agree 

slicfrtly 

strongly 

agree 

agree 

nor disagree 

dsacyee 

dsagree 

2. Younger pilots catch on to automation faster than older ones. 


A 

B 

C 

D 

E 

strong y 

sligfrtly 

neither agree 

slightly 

strongly 

agree 

agree 

nor disagree 

dsagree 

dsagree 

3. Flying today is more challenging than ever. 



A 

B 

C 

D 

E 

strongly 

slightly 

neither agree 

slightly 

strongly 

agree 

agree 

nor disagree 

dsacyee 

dsagree 

4. The FMS/CDU is easy to use in 

normal line fiying. 



A 

B 

C 

D 

E 

strongly 

siigfrtiy 

nether agree 

slightly 

strongly 

agree 

agree 

nor disagree 

dsagree 

disagree 

6. 1 think they've gone too far with automation. 



A 

B 

C 

D 

E 

strorrfy 

slightly 

nether agree 

slightly 

strongly 

eg-ee 

agree 

nor disagree 

dsagree 

disagree 
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6. Auto! and capability definitely enhances safety. 

ABC 


strong y 
a grae 


slightly 

agree 


neither erjee 
nor dsagyee 


D 

slightly 

dsagree 


E 

strosx^y 

dsagree 


7. I spend more time setting up and managing the automatics (such as the 

FIViS/CDU) than 1 wouid hand flying or using the old style autopilots. 

ABODE 
strongly slightly neither cg-ea slightly strongly 

agree agree nor dsagree dsagree dssgyee 

8. I like to use the new features of the 767 as much as possible. 

•-> A B C D E 

stroncfy slightly neither agree slightly strongly 

agree agree nor dsagree dsagree dsagree 


9. I am worried about sudden failures of the new devices like the FMS Com- 
puter and the CRT displays. 


A 

strong y 

agree 


B 

slightly 

agree 


neither agree 
nor dsagree 


D 

slightly 

dsagree 


E 

strongly 

dsagree 


1 0. I use automatic devices a lot because i find them useful. 

A B C D E 

strongly slightly neither rrree slightly strongly 

agree agree nor dsagree dsagree dsagree 

11. I enjoy flying the 7S7 more than the older aircraft. 

A B C D E 

strongly slightly neither agree slightly strortcjy 

og-ee agree nor dsagree dsagree dsagree 

12. I always know what mode the Autopilot/Flight Director is in. 

A B C D E 

strongly slightly neither agree slightly strondy 

agree agree nor dsagree dsagree dsagree 


13. I can fly as efficiently as the FMS without its help. 


A 

strongly 

agree 


B 

slightly 

agree 


neither agree 
nor dsagree 


D 

slightly 

dsagree 


E 

stroncty 

dsagree 


61 


14. 


( hand fly part of every trip to keep my skills up. 


A 

strongly 

agree 


B C 

sligitiy neither agree 

agree nor dsagree 


0 

snyiti', 

disagree 


E 

siron^y 

disagree 


15. 


Automation does not reduce workload, since there is more to keep vstch 
over. 


A 

strongly 

ayee 


BCD 
slightly neither acyee siic^Ttly 

agree nor dsagree dsagree 


E 

strongly 

dsacyee 


16. I can find the exact location of important controls and switches without any 


hesitation. 

A 

B 

c 

D 

E 

strongly 

slightly 

neither agree 

sligitly 

strongly 

agree 

agree 

Hu r d sagr ee 

cfsagree 

disagree 


17. Automation is the thing that is going to turn my company around and make it 

profitable again. 

A B C D E 

strongly slightly neither agree slightly strongly 

a$yee agree nor dsacyee disagree disagree 

18. Pilots who overuse automation will see their flying skills suffer. 

ABODE 
strongly slightly neither acyee sligitly strongly 

agree agyee nor dsagree dsagree disagree 

19. The AD! and EHSi displays are always iegibie and easy to read. 


A 

B 

C 

D 

E 

strongly 

siichtly 

neither agree 

slightly 

stroncfy 

agree 

agree 

nor dsagree 

dsagree 

disagree 


20. I am favorable toward automation in the cockpit - the more the better. 


A 

B 

C 

D 

E 

strongly 

slightly 

neither agree 

slightly 

strongly 

ogees 

agree 

nor dsagyee 

dsacyee 

disagree 

21. Flying the 767 is 

definitely easier than flying other aircraft. 


A 

B 

C 

D 

E 

strongly 

slightly 

neither agree 

slightly 

strongly 

agree 

agree 

ror dsagree 

dsagree 

dsacyee 

$ 
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22. 

Setting piloting priorities with 

this new cockpit technology is no more diffi- 


ait than in our other airplanes. 




V A 

B 

C 

0 

E 


strongly 

digitiy 

neither ogres 

dic^rtiy 

strongy 


agree 

ogres 

nor cSsagree 

dsagee 

dsagee 

23. 

w 

Wa should have full autcthrotties on all the company'3 aircraft. 


A 

B 

C 

D 

E 


strongfy 

slightly 

neither agree 

shortly 

strongly 


agree 

ugrea 

ror disagree 

dsagee 

cfsagee 

24. 

Automation frees 

me of much of the routine, mechanical parts 

of flying so 


can concentrate more on "managing" the 

flight. 



A 

B 

C 

D 

E 


strorr^y 

slightly 

neither agree 

slightly 

stron^y 


agee 

agree 

nor dsagee 

dsagee 

dsagee 

25. 

1 have serious concerns about 

the reliability of 

this nsw equipment. 


A 

B 

C 

0 

E 


strondy 

sli^rtly 

neither scree 

sllcfitly 

strongly 


agree 

ag-ee 

nor dsagea 

dsagea 

dsagee 

26. 

Sometimes what the automatics do or don't do 

takes me by surprise. 


A 

B 

C 

D 

E 


strong y 

shortly 

neither agree 

shortly 

strongly 


agoe 

ajyee 

nor dsagrae 

dsagea 

dsagee 

27. 

It is easier to cross-check the other pilot in the 767 than in our other air 


planes. 






A 

B 

C 

D 

E 


strongly 

shortly 

neither agree 

dl^itiy 

stron^y 


agree 

agree 

nor dsag-ee 

dsagee 

dsagee 

28. 

Too much automation can be dangerous. 




A 

B 

C 

D 

E 


strongly 

slightly 

neither agree 

slightly 

strongly 


ogres 

agree 

nor dsagea 

dsagee 

dsagee 

29. 

The new equipment is more reliable than the old. 



A 

B 

C 

D ' 

E 


stronc^y 

sli^itiy 

neither agree 

di^itly 

strorx^y 


agree 

ag-ee 

nor dsagee 

dsagee 

dsagee 
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30. It is important to me to fiy the most modem plane in the company's fleet. 



A 

B 

C 

D 

E 


strongly 

slightly 

neither agree 

slightly 

strongly 


agree 

agr se 

O 

nor cisagrea 

dsagree 

dsagree 

31. 

1 am concerned about a possible loss of my flying skills with too much auto 


motion. 






A 

B 

C 

D 

E 


strongly 

slightly 

neither agree 

slightly 

strongly 


agree 

"5 

agree 

nor dsagree 

dsagree 

dsagree 

32. 

Automation reduces overall workload. 




A 

B 

C 

D 

E 


strongly 

slightly 

natter agree 

slightly 

strongly 


agree 

ayoe 

nor dsagree 

dsagree 

dsagree 

33. 

1 always feel 1 am ahead of the airplane. 




A 

B 

C 

D 

E 


strongly 

slightly 

naitter agree 

slightly 

strongly 


agree 

agree 

nor dsagree 

dsagree 

dsagree 

34. 

Hand flying is the part of the 

trip 1 enjoy most. 




A 

B 

C 

D 

E 


strongly 

slightly 

neither agree 

slightly 

strongly 


agree 

agree 

nor dsagree 

dsagree 

dsagree 


35. I use automatic devices mainly because the company wants me to. 

A B C D E 

strongly slightly neither ag-ee slighitly strongly 

agree agree nor disagree dsagree disagree 


36. The FMS/CDU requires little or no in-flight button-pushing below FL180. 


A 

strongly 

acyoe 


BCD 
slightly neither agree slightly 

agree nor dsagree disagree 


E 

strongly 

disagree 
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Appendix B 


Frequency-of-Use Table 


The frequency of use table was distributed in two forms (..see Appendix A) because of some 
ambiguities in the instructions and some apparent inconsiste. cies in the response to <.ne table. 
The second version of the table attempted to resolve the ambiguities by explicitly showing the 
four mutually exclusive AFDS modes: CMD (command), CWS (control wheel steering), 
Hand-flv with FD (Flight Director), and Hand-fly without FD. Both versions had these 
instructions in common: 


For each phase-of -flight column in the table, enter the percent of your legs on 
which you use the feature described on the left—" 

For both versions, the pilots (correctly) interpreted the instructions in one of two ways. 
About 75% of the pilots filled in numbers for which the percentages add up to 100 (the 100% 
group), and the remaining pilots used numbers that added up to more than 100% (the greater- 
than-100% group). The ambiguity arises because more than one feature can be used during a 
particular phase of flight. In fact, several pilots reported (in the margins) hand-flying to 
10000 ft and then engaging the CMD mode. The 100% group reported this as 25% hand flying, 
and 75% CMD, whereas the the other group would report this as 100% for each teature in 
this phase of flight. Because of these different reporting styles, the two groups (the 100% and 
greater-than-100%) have their use of the autopilot modes reported separately. 

The data were reduced to a common format for the autopilot use as follows: for the first ver- 
sion of the table that did not explicitly contain a CMD row, a CMD row was created bv 
adding up the use for the other three modes and subtracting the sum from 100: thus this 
should be considered a lower bound to the use that would have been reported had the CMD 
row been in the .able. 

The results are summarized in Table B. Since percentages are being reported, there is a sub- 
stantial "floor" and "ceiling" effect, and a traditional measure of means and standard devia- 
tions are meaningless. Each cell in the table reports the quartile scores (Ql, Q2, and Q3) and 
the number of valid responses in each cell; these quartile points divide the valid responses into 
four equal sized groups; thus, one-quarter of the responses fall below the level in the Ql row; 
one-half fall below the level in the Q2 row; and three-quarters of the responses fall below 
the level in the Q3 row. For example, in the greater-than-100% group, there were 23 valid 
responses for the "hand-fly with flight director" mode during the "Transition and enroute 
climb" phase of flight: one-quarter of the responses (about 6) fell below 52.5 %; one-half of 
the responses (about 12) fell below 90%; and three-quarters of the responses (about 18) fell 
below 93.75%. Linear interpolation is used on the percent of legs and the number of responses 
to calculate the quartile points. 


- 65 - 


Table B. Frequency of Use 


V 






PHASE CF FLIGHT 





Takeoff 

Transi- 

Cruise 

Descent 

Termi- 

Final 

Land- 

FEATURE 


to 1000 

tion & 


(Cruise 

nal 

Ap- 

ing 



AGL 

En- 


to 

Area 

proach 





rouie 


10,000 







Climb 


MSL) 





GREATER THAN 100% GROUP 


Qi 

0.0 

52.5 

100.0 

82.5 

50.0 

25.0 

0.0 

CMD 

Q2 

0.0 

90.0 

100.0 

95.0 

75.0 

50.0 

10.0 

03 

0.0 

93.75 

100.0 

100.0 

87.5 

50.0 

20.0 


N 

18 

23 

23 

23 

23 

23 

22 


01 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

CWS 

02 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Q3 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 


N 

27 

29 

28 

29 

29 

29 

26 

HAND 

FLY:NO 

FD 

01 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

02 

Q3 

0.0 

25.0 

2.0 

50.0 

0.0 

17.5 

0.0 

45.0 

17.5 

40.0 

17.5 

40.0 

10.0 

32.5 

N 

25 

25 

23 

24 

24 

24 

20 


01 

68.75 

10.0 

0.0 

5.0 

20.0 

50.0 

50.0 

HAND 

Q2 

100.0 

50.0 

2.5 

20.0 

60.0 

75.0 

82.5 

FLY:FD 

Q3 

100.0 

94.0 

80.0 

80.0 

80.0 

88.25 

98.0 


N 

29 

28 

26 

26 

26 

25 

22 


100% GROUP 


01 

0.0 

20.0 

96.5 

80.0 

50.0 

20.0 

0.0 

:md 

Q2 

0.0 

55.0 

100.0 

90.0 

67.0 

40.0 

5.0 

Q3 

0.0 

90.0 

100.0 

100.0 

82.5 

50.0 

10.0 


N 

53 

70 

68 

67 

68 

67 

65 


Qi 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

cws 

02 

03 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 


N 

64 

70 

70 

69 

70 

70 

65 

HAND 

FLY:NO 

FD 

01 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Q2 

03 

N 

0.0 

10.0 

59 

0.0 

5.0 

56 

0.0 

0.0 

52 

0.0 

0.0 

52 

0.0 

6.25 

53 

0.0 

10.0 

57 

5.0 

45.0 

56 


01 

90.0 

15.0 

0.0 

0.0 

15.0 

50.0 

65.5 

HAND 

Q2 

100.0 

50.0 

0.0 

10.0 

30.0 

55.0 

90.0 

FLY:FD 

Q3 

100.0 

80.0 

2.75 

20.0 

50.0 

77.5 

100.0 

1 

N 

68 

66 

61 

61 

66 

68 

64 
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Table B. Frequency of Use (con) 


FEATURE 


0 

PHASE OF FLIGHT 



Takeoff 
lo 1000 
AGL 

Transi - 
tion & 
En - 
route 
Climb 

Cruise 

Descent 

(Cruise 

to 

10.000 

MSL ) 

Termi - 

nal 

Area 

Final 

Ap- 

proach 

Land - 

ing 

VERTI- 

01 

0.0 

0.0 

0.0 

225 

20.0 

0.0 

0.0 

CAL 

02 

0.0 

10.0 

0.0 

50.0 

50.0 

0.0 

0.0 

SPEED 

03 

0.0 

30.0 

0.0 

85.0 

80.0 

20.0 

0.0 

N 

54 

89 

44 

92 

89 

64 

32 

n A KK 

01 

1 0.0 

50.0 

62.5 

50.0 

23.75 

2.5 

0.0 

LIMIT 

02 

90.0 

90.0 

100.0 

100.0 
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Table B. Fre^tncy of Use (con) 
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